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Abstract 

Introduction: Veno-venous extracorporeal membrane oxygenation (vvECMO) can be a life-saving therapy in patients 
with severe acute lung failure refractory to conventional therapy. Nevertheless, vvECMO is a procedure associated with 
high costs and resource utilization. The aim of this study was to assess published models for prediction of mortality 
following vvECMO and optimize an alternative model. 

Methods: Established mortality risk scores were validated to assess their usefulness in 304 adult patients undergoing 
vvECMO for refractory lung failure at the University Medical Center Regensburg from 2008 to 2013. A parsimonious 
prediction model was developed based on variables available before ECMO initiation using logistic regression 
modelling. We then assessed whether addition of variables available one day after ECMO implementation 
enhanced mortality prediction. Models were internally validated and calibrated by bootstrapping (400 runs). 
Predictive ability, goodness-of-fit and model discrimination were compared across the different models. 

Results: In the present study population, existing mortality prediction tools for vvECMO patients showed suboptimal 
performance. Evaluated before vvECMO initiation, a logistic prediction model comprising age, immunocompromised 
state, artificial minute ventilation, pre-ECMO serum lactate and hemoglobin concentrations showed best mortality 
prediction in our patients (area under curve, AUC: 0.75). Additional information about norepinephrine dosage, fraction 
of inspired oxygen, C-reactive protein and fibrinogen concentrations the first day following ECMO initiation further 
improved discrimination (AUC: 0.79, P = 0.03) and predictive ability (likelihood ratio test, P< 0.001). When classifying 
patients as lower (<40%) or higher (>80%) risk based on their predicted mortality, the pre-ECMO and day1-on-ECMO 
models had negative/positive predictive values of 76%/82% and 82%/81%, respectively. 

Conclusions: While pre-ECMO mortality prediction remains a challenge due to large patient heterogeneity, evaluation 
one day after ECMO initiation may improve the ability to separate lower- and higher-risk patients. Our findings support 
the clinical perception that chronic health condition, high comorbidity and reduced functional reserves are strongly 
related to survival during and following ECMO support. Renewed evaluation the first day after ECMO initiation may 
provide enhanced guidance for further handling of ECMO patients. Despite the usefulness of prediction models, 
thorough clinical evaluation should always represent the cornerstone in decision for ECMO. 
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Introduction 

For more than 40 years, veno-venous extracorporeal mem- 
brane oxygenation (wECMO) has been used in critically ill 
patients with reversible acute lung failure (ALF). Following 
major improvements in ECMO technology and safety, the 
survival benefit shown in the CESAR trial [1], the useful- 
ness of ECMO support during the influenza- A/HlNl-virus 
epidemic in 2009 to 2010 [2] and the increasing evidence 
that early ECMO may help to avoid substantial lung and 
consecutive organ injury [3-5], ECMO is now implemented 
more frequently and in a broader spectrum of patients. 

Objective risk assessment is a valuable tool to aid the 
selection of therapeutic approach, to compare ECMO 
practice and success rates across different ECMO cen- 
ters and to provide relatives with objective information 
about the patient's condition. Thus, there has been an 
increasing interest in identifying important prognostic 
factors for wECMO. Published work includes two risk 
scores designed for pre-ECMO mortality prediction in 
patients with acute respiratory distress syndrome (ARDS), 
the ECMOnet score [6] and the PRESERVE score [7]. 
Presently, it is thought that multi-organ failure and 
sepsis play a more important role than the patient's 
hemodynamic and respiratory state in the acute situation 
[6,8,9]. However, findings have been inconsistent. Limi- 
tations of previous studies include changes in ECMO 
technology and procedure over time, small study popu- 
lations, large patient heterogeneity, limited registration 
of potentially important factors related to outcome, and 
application of statistical methods which may not take 
into account the random variability of patients, thus 
leading to results that have not been reproducible. 

A large study population, application of up-to-date 
statistical methods designed to avoid overfitting and 
use of thorough internal and external validation are 
crucial steps in the development of reproducible ECMO 
risk scores to be integrated in clinical practice. Based 
on prospectively collected data from an institution where 
different forms of ECMO are now performed more than 
100 times per year, we validated the existing risk scores 
and compared their predictive ability with the more 
general Sequential Organ Failure Assessment (SOFA) 
score frequently used in the intensive care unit (ICU). 
Furthermore, in accordance with the hypothesis that 
ECMO outcome is largely dependent on the patient's 
extra-pulmonary status and general health condition, our 
aim was to examine data regarding clinical characteristics, 
respiratory and laboratory parameters, to develop two 
novel risk scores: one designed for pre-ECMO mortality 
prediction and one including parameters measured 
one day after ECMO implantation. We compared the 
different pre-ECMO mortality prediction models and 
investigated whether day 1 parameters could enhance 
predictive ability. 



Materials and methods 

From 1 January 2008 to 15 July 2013, wECMO was ini- 
tiated in 304 consecutive adults with ALF refractory to 
conventional therapeutic modalities at the University Med- 
ical Center Regensburg (UKR), Germany. The general 
indication for wECMO was either a ratio of partial arter- 
ial pressure of oxygen/fraction of inspired oxygen (Pa02/ 
Fi02) <80 mmHg on a positive end- expiratory pressure 
of >16 cm H 2 0, or a refractory respiratory acidosis 
(pH <7.25), despite optimization of conservative therapy. 

The ECMO circuit consisted of a centrifugal pump and 
a coated polymethylpentene oxygenator. In 99% of the 
patients, cannulation was performed percutaneously with 
Seldinger's technique. In 75%, single-lumen cannulas were 
implemented with blood-flow from the femoral vein and 
to the jugular (80%), subclavian (15%) or contralateral 
femoral (4%) vein. Nineteen percent and six percent of the 
patients received a single double-lumen jugulo-jugular 
(Avalon Elite 8 , MAQUET Holding B.V. & Co KG, Rastatt, 
Germany) and femoro-femoral (Novaporf twin, Novalung 
GmbH, Heilbronn, Germany) cannula, respectively. Fur- 
ther ECMO settings and patient management followed an 
institutional protocol as described previously [8] . 

Clinical data regarding pre-, intra- and post-ECMO char- 
acteristics and functions were prospectively registered 
in the UKR ECMO database. More detailed method 
descriptions are given in Additional file 1. All patients 
were followed up until in-hospital death (non-survi- 
vors) or hospital discharge (survivors). The study was 
approved by the local ethical committee of the UKR 
(reference number: 14-180-0051). The requirement for 
individual patient consent was waived as this study was 
based on anonymized data from routine care. 

Statistical analysis 

Data are given as median (95% confidence intervals; CI) 
for continuous variables, and as frequency (percentage) for 
categorical variables. The Mann Whitney U-test and x 2 test 
were used to investigate differences between survivors and 
non-survivors for continuous and categorical variables, 
respectively. All tests were two-sided and P-values <0.05 
were considered significant. Cases with missing values 
were excluded from the analyses. Statistical analyses were 
performed using SPSS (version 21.0, SPSS Inc., Chicago, 
IL, USA), Stata (version 12.1, StataCorp LP, College 
Station, TX, USA), SigmaPlot (version 12.0, Systat Soft- 
ware, Inc., San Jose, CA, USA), Minitab (version 15.1.30, 
Minitab Inc., State College, PA, USA) and the 'rms', 
'Hmisc', 'xtable' and 'party' packages of R statistical soft- 
ware (version 3.0.1, R Foundation 3 ). 

Validation of external risk scores 

The external scores were tested in the present patient 
population as described in the original publications (see 
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Additional file 2). SOFA was scored for each patient 
during data collection and stored in the database. ECM- 
Onet and PRESERVE scores were calculated retrospect- 
ively. We found matching definitions for all variables 
except plateau pressure, which was not registered in 
our database due to the mode of mechanical ventilation 
used. Peak inspiratory pressure was used as a substitute. 
The discriminative abilities of the external risk scores were 
assessed by the area under the receiver-operating charac- 
teristics curves (AUC). Since poor fit may be explained 
by substantial differences in the study populations, pre- 
dictors identified by the two scores were recalibrated 
using multivariate logistic regression analyses. The original 
estimates for the linear predictor variables, on which the 
additive ECMOnet and PRESERVE scores were based, 
were compared with the recalibrated coefficients and odds 
ratios, respectively. Clinical usefulness was evaluated using 
information from the original studies. For the ECMOnet 
score, we compared the sensitivity and specificity in our 
population using the authors' proposed cut-off score for 
mortality of >4.5 as well as the estimated optimal cut-off 
in our population, calculated by the Youden index [10]. 
For the PRESERVE score, we compared mortality rates 
across groups of PRESERVE scores as described in the 
original study. We used a simple prognostic separation 
index (PSEP) to estimate the ability of the score to separ- 
ate individuals into prognostic groups and compared the 
index across the cohorts to evaluate the reproducibility 
in new patients [11]. 

Model 1: Pre-ECMO mortality prediction 

Variable selection for prediction of in-hospital death com- 
bined literature review, clinical experience and hypotheses 
of potential influences on ECMO outcome as well as 
computer-based modelling (Additional file 3). A final 
parsimonious model was derived using logistic regression 
on the full dataset (n = 304), and further validated using 
bootstrap resampling with replacement (400 runs). Pre- 
dicted probabilities of mortality were calculated based on 
bootstrapped coefficients (400 runs). Model performance 
was evaluated by plotting observed probabilities across 
deciles of predicted risk. Model discrimination was 
assessed with AUC, goodness-of-fit with a Hosmer- 
Lemeshow (HL) test, model overfitting by calculation of 
the shrinkage factor, and calibration by a calibration plot. 
The novel risk model was compared with the ECMOnet, 
PRESERVE and SOFA scores (Additional file 1). 

Model 2: Mortality prediction on day 1 

For the second model, we added variables available one 
day after ECMO initiation to the final pre-ECMO model. 
The procedure of variable selection, validation and cali- 
bration of the final model was performed as described 
for Model 1. Models 1 and 2 were compared statistically 



by a likelihood ratio (LR) test. A simultaneous comparison 
of discrimination for all models (n = 241) was performed 
using the method for non-parametric correlated AUCs as 
proposed by DeLong et al. [12]. The novel day 1 risk 
model was also compared with the ECMOnet, PRESERVE 
and SOFA scores calculated with day 1 data (n = 236). 

Clinical usefulness of the two novel models was evalu- 
ated by dividing the cohort into three groups based on 
predicted mortality risks: low (<40%), moderate (40% to 
80%) and high (>80%) (Additional file 1). A simple PSEP 
was calculated to address separation between the lowest 
and highest risk groups. Negative and positive predictive 
values were calculated for the lowest and highest risk 
groups, respectively, as a measure of clinical validity. 

Results 

Study population 

Overall survival until hospital discharge was 61.5%; 81 
patients (27%) died during mechanical support and 36 
patients (12%) died after weaning from ECMO. The best 
outcome was noted in trauma patients (74%), followed by 
primary lung failure (66%), extra-pulmonary lung failure 
and other causes (both 50%). Before ECMO initiation, 
non-survivors were longer hospitalized, had higher serum 
lactate concentrations, lower hemoglobin concentrations 
and more frequently showed signs of additional organ 
dysfunctions (pre-ECMO need for continuous veno-venous 
hemofiltration, higher bilirubin and liver enzyme concen- 
trations). Pa02/Fi02, commonly used in the classification 
of ARDS severity, did not differ significantly between 
survivors and non-survivors. On the first day on ECMO, 
non-survivors showed less improvement of their respi- 
ratory failure, with higher minute ventilation, persisting 
low oxygenation and higher norepinephrine dosages. A 
comparison of survivors and non-survivors is given in 
Table 1 and Additional file 4. 

Performance of external risk prediction models in 
our population 

Both the ECMOnet and PRESERVE scores showed limited 
discrimination (AUC <0.70) and poor goodness-of-fit in 
our population (Table 2). Recalibration of the predictor 
variables of the ECMOnet score revealed substantial dif- 
ferences in the coefficients of the validation and original 
study cohorts (data not shown). Discrimination was not 
improved by recalibration. The proposed cut-off score for 
mortality >4.5 yielded a sensitivity and specificity of 58% 
and 53%, respectively. The optimal cut-off for mortality in 
our population was >5.0, giving a sensitivity and specificity 
of 52% and 65%, respectively. For the PRESERVE score, 
the observed mortality rates for score classes 0 to 2, 3 to 
4, 5 to 6 and >7 showed an increasing trend as in the 
original study; however, the rates were higher for the 
lower-risk groups and lower for the higher-risk groups 
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Table 1 Comparison of survivors and non-survivors after ECMO 



Non-survivors (n = 1 1 7) 



Survival to discharge (n = 187) 



P-value 



Baseline clinical characteristics 

Age (years) 

Immunocompromised state 3 
Sequential organ failure assessment score 
Continuous veno-venous hemofiltration pre-ECMO 
Classification of acute lung injury b 

- Group 1: Pulmonary 

- Group 2: Extra-pulmonary 

- Group 3: Trauma 

- Group 4: Others 
Pre-ECMO duration (days) of 

- Hospitalization 

- Mechanical ventilation 

Baseline cardiorespiratory parameters 

Minute ventilation (L/minute) 

Baseline laboratory parameters 

Lactate (mmol/L) 

Lactate dehydrogenase (U/L) 

Bilirubin (mg/dL) 

ASAT (U/L) 

Hemoglobin (g/dL) 

D-dimer (mg/L) 

Procedural characteristics of ECMO treatment 

Transport-ECMO (no/yes) 

Red cell transfusions (about 320 mL) per patient 

Plasma transfusions (about 250 mL) per patient 

Platelet transfusions (about 270 mL) per patient 

Laboratory and cardiorespiratory status after one 

Minute ventilation (L/minute) 

Tidal volume (mL) 

Norepinephrine infusion (ug/minute/kg) 
Blood gas analysis 

- Fi0 2 (%) 

- Pa0 2 /Fi0 2 (mmHg) 
Laboratory parameters 

- Lactate (mmol/L) 

- C-reactive protein (mg/L) 

- Bilirubin (mg/dL) 

- Lactate dehydrogenase (U/L) 

- International normalized ratio 



54 (50 to 57) 
37 (31.6%) 

13 (12 to 14) 
45 (385%) 

56 (47.9%) 
36 (30.8%) 
10 (8.6%) 
15 (12.8%) 

10 (8 to 12) 
5 (3 to 7) 

11.2 (10.6 to 12.0) 

38 (30 to 47) 
510 (446 to 608) 
1.2 (0.9 to 1.6) 
1 22 (96 to 1 59) 

9.8 (9.4 to 1 0.3) 

11 (9 to 14) 

34 (29.1%) 
8 (6 to 9) 
3 (2 to 5) 
1 (Oto 1) 

day on ECMO 

5 (5 to 6) 
305 (278 to 332) 
0.18 (0.13 to 0.25) 

65 (60 to 68) 
122 (112 to 133) 

36 (27 to 47) 
162 (143 to 180) 

1.9 (1.5 to 2.3) 
584 (502 to 717) 
1.38 (1.32 to 1.48) 



46 (43 to 48) 
26 (13.9%) 

11 (11 to 12) 
36 (19.3%) 

107 (57.2%) 
36 (19.3%) 
29 (15.5%) 
1 5 (8.0%) 

5 (4 to 7) 

2 (2 to 3) 

104 (10.0 to 10.9) 

23 (20 to 27) 
452 (408 to 499) 

0.9 (0.8 to 1.1) 
78 (65 to 96) 
10.9 (10.5 to 11.3) 
8 (7 to 1 0) 

99 (52.9%) 

3 (3 to 4) 
0 (0 to 1) 
0 (0 to 0) 

4 (4 to 5) 
278 (262 to 294) 
0.12 (0.09 to 0.14) 

55 (52 to 58) 
146 (137 to 155) 

24 (21 to 28) 
214(196 to 231) 

1.1 (1.0 to 1.3) 
511 (462 to 573) 
1.27 (1.24 to 1.31) 



<0.001 
<0.001 
0.001 
<0.001 
0.022 



0.023 
0.013 

0.038 

0.002 
0.076 
0.060 
0.005 
<0.001 
0.022 

<0.001 
<0.001 
0.001 
<0.001 

0.011 
0.097 
0.041 

<0.001 
<0.001 

0.002 
<0.001 
0.015 
0.054 
0.002 



Continuous variables are shown as median (95% confidence interval), categorical variables as n (%}. Differences for survivors and non-survivors are shown for 
P <0.1. A wider comparison can be seen in Additional file 4. immunocompromised state included hematological malignancies, solid tumors, solid organ transplantation, 
high-dose or long-term corticosteroid or other immunosuppressive therapy, or human immunodeficiency virus infection. b Group 1 : primary lung failure, including bacterial, 
viral, fungal or aspiration pneumonia; Group 2: extra-pulmonary sepsis with secondary lung injury; Group 3: multiple trauma with ARDS; Group 4; other 
pathologies, including near drowning, chronic lung diseases, such as lung fibrosis and lung transplantation. ASAT, aspartate aminotransferase; ECMO, 
extracorporeal membrane oxygenation; Fi0 2 ,fraction of inspired oxygen; Pa0 2 /Fi0 2 , ratio of partial arterial oxygen pressure/fraction of inspired oxygen; 
SOFA, Sequential Organ Failure Assessment. 
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Table 2 Comparison of mortality prediction models in the study population (number = 304) 



Model 



Valid n 



AUC 



P-value a 



Hosmer-Lemeshow test P-value 



Pre-ECMO prediction 

SOFA score 
ECMOnet score 
PRESERVE 
Model 1 

Day 1 prediction 

Model 2 



303 
280 



284 



263 



0.611 (0.544-0.678) 
0.604 (0.537-0.671) 
0.685 (0.623-0.748) 
0.746 (0.689-0.804) 

0.786 (0.730-0.843) 



0.027 
0.009 
0.12 
Reference 

0.026 



<0.0001 b 
0.0002 b 
0.004 b 
0.50 



0.73 



a Pairwise comparisons of AUCs with Model 1 using DeLongs method (n = 241 ) [1 2]. b Modified Hosmer-Lemeshow test comparing grouped mortality risk against 
the deciles of Model 1 predictions. AUC, area under receiver operating characteristics curve; ECMO, extracorporeal membrane oxygenation; SOFA, Sequential 
Organ Failure Assessment. 



in the present study population (Table 3). PSEP was 
reduced from 0.81 in the original analysis to 0.53, 
although the overall mortality rates were similar. Recali- 
bration gave minimal differences in the coefficients of the 
validation and original study cohort (data not shown). 
Neither the ECMOnet nor the PRESERVE score showed 
enhanced discrimination compared to the SOFA score 
(P = 0.67 and P = 0.25, respectively, Figure 1). 

Novel risk prediction models 
Model 1 

The final pre-ECMO mortality prediction model con- 
tained five variables (Table 4). A plot of observed mor- 
tality against increasing deciles of predicted mortality 
risk showed good predictive ability, although the 
model tended to overestimate mortality in patients 
with predicted risk between 18% and 23% (third decile 
group, Additional file 5). The prediction model showed 
acceptable discrimination (AUC >0.70 but <0.80), a 
good goodness-of-fit (HL test, P = 0.50) and had a 
shrinkage factor of 0.83. 

Comparison of pre-ECMO prediction models The 

mortality risk predicted by the external scores did not 
adequately fit with the observed mortality (modified HL 
test, P <0.01 for all scores, Table 2). By analysis of AUC, 
Model 1 showed enhanced discrimination compared to 



the SOFA and ECMOnet score (P = 0.03 and P = 0.01, 
respectively, Table 2, Figure 1). However, it did not im- 
prove discrimination compared to the PRESERVE score 
(P =0.12). 



Model 2 

Addition of day-1 parameters resulted in a predictive 
model based on eight variables (Table 4). Model 2 showed 
good predictive ability, acceptable discrimination and 
adequate goodness-of-fit (Table 2). The calibration plot 
showed good agreement between predicted and observed 
probability (Additional file 5). The shrinkage factor was 
0.87. Model 2 had significantly better predictive ability 
than Model 1 (LR-test: P <0.001) and showed enhanced 
discrimination (Figure 1) compared both to Model 1 and 
the PRESERVE score (P = 0.03 and P = 0.003, respectively). 
In a separate nested analysis (n = 236), Model 2 (AUC: 
0.794 (0.736 to 0.853)) also showed a better discriminative 
ability compared to the SOFA (AUC: 0.588 (0.512 to 0.665)), 
ECMOnet (AUC: 0.609 (0.536 to 0.683)) and PRESERVE 
(AUC: 0.699 (0.629 to 0.768)) scores calculated using day 
1 data (P <0.001, P <0.001 and 0.01, respectively). 

Classification of patients into three respective risk groups 
revealed good agreement between mean predicted and 
observed group mortality risks for both Models 1 and 2 
(Table 5). 



Table 3 Performance of the PRESERVE score in the original and validation cohorts 



PRESERVE 
score class 

0 to 2 
3 to 4 
5 to 6 
>7 
Total 
PSEP 



Original study population 



Number (%) Mortality rate up to six months post-ICU 

34 (25.0%) 3% 
38 (27.9%) 21% 
26(19.1%) 46% 
38 (27.9%) 84% 
136 (100.0%) 40% 



UKR ECMO registry 



Number (%) 

35 (12.1%) 
90 (31.1%) 
97 (33.6%) 
67 (23.2%) 
289 (1 00.0%) 



In-hospital mortality rate 

11% 

28% 
40% 
64% 
38% 

53% (95% CI 37-69) 



a The 95% CI could not be calculated. CI, confidence interval; ECMO, extracorporeal membrane oxygenation; PSEP, prognostic separation index; UKR, University 
Medical Center Regensburg. 
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1.0 



0.8 



>. 0.6 



f> 0.4 



0.2 



0.0- 




Model 2 (Day 1) 
Model 1 (Pre-ECMO) 
PRESERVE 
ECMOnet 
SOFA 



0.0 



0.2 



0.8 



1.0 



0.4 0.6 
1 - Specificity 

Figure 1 Comparison of the receiver-operating curves for all 
risk prediction tools (n = 241). Neither the ECMOnet nor the 
PRESERVE score had significantly better discrimination compared to 
the SOFA score (P = 0.67 and 0.25, respectively). Model 1 improved 
discrimination compared to the SOFA and the ECMOnet score (P = 0.03 
and 0.009, respectively). Addition of parameters available one day after 
ECMO initiation further enhanced discrimination compared to both 
Model 1 and the PRESERVE score (P = 0.03 and P= 0.003, respectively). 
Further statistical comparison is given in Table 2. ECMO, extracorporeal 
membrane oxygenation; SOFA, Sequential Organ Failure Assessment. 



Discussion 

In this large study regarding mortality prediction in ALF 
patients with vvECMO support, we found that both 
the PRESERVE and ECMOnet score were suboptimal 
to predict mortality accurately in our population. Two 
novel risk models were developed showing improved 
predictive ability. The addition of parameters available 
the first day after ECMO initiation enhanced mortality 
prediction. The combination of predictors found to be 
associated with the most optimal risk prediction in this 
study add evidence to the hypothesis that high comorbid- 
ity and unresponsive respiratory failure are important 
determinants of mortality following wECMO support. 

Pre-existing tools for predicting mortality 

The SOFA score was designed for and has become 
well-integrated in clinical practice as an easily available 
bedside tool to evaluate organ failure/dysfunction over 
time in ICU patients. As recently described, a high pre- 
ECMO SOFA score has been associated with a higher 
mortality risk [13]. 

The ECMOnet score is an additive score (range 0 to 10) 
based on baseline characteristics of 60 patients with severe 
ARDS due to suspected or confirmed HlNl-influenza 
virus infection [6]. In our study population, the ECMOnet 
score did not show better discrimination than the SOFA 
score. Comparison of original and recalibrated estimates 
revealed considerable differences between the study popu- 
lations, but continued poor discrimination after recalibra- 
tion indicated that the combination of predictors was not 



Table 4 Novel mortality prediction models for ALF-patients receiving ECMO support 



Coefficient 



OR 



95% CI 



Model 1 (pre-ECMO) 

Age (per five years) 0.176 1.193 

mmunocompromised state 0.958 2.605 

Minute ventilation (L/minute) 0.098 1.103 

Pre-ECMO hemoglobin (g/dL) -0.182 0.834 

Pre-ECMO lactate (mmol/L) 0.013 1.013 

Intercept -2.083 
Model 2 (Day 1) 

Age (per five years) 0.184 1.202 

mmunocompromised state 1.093 2.984 

Minute ventilation (L/minute) 0.137 1.147 

Pre-ECMO hemoglobin (g/dL) -0.208 0.812 

Day 1 Fi0 2 (per 1 0%) 0.264 1 .302 

Day 1 fibrinogen (mg/dL) -0.002 0.998 

Day 1 norepinephrine (ug/minute/10 kg) 0.159 1.172 

Day 1 C-reactive protein (mg/L) -0.004 0.996 

Intercept -1.893 

ALF, acute lung failure; CI, confidence interval; ECMO, extracorporeal membrane oxygenation; Fi0 2 , fraction of inspired oxygen; OR, odds ratio. 



(1.148 to 1.239) 
(1.316 to 5.158) 
(1.014 to 1.199) 
(0.728 to 0.954) 
(1 .004 to 1 .023) 



(1.148 to 1.258) 
(1.394 to 6.391) 
(1.030 to 1.276) 
(0.696 to 0.947) 
(1.232 to 1.376) 
(0.996 to 0.999) 
(0.980 to 1.401) 
(0.992 to 0.999) 
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Table 5 Clinical validation of novel mortality scores 


Predicted risk 


N 


Mean predicted risk 


Observed mortality risk 


Clinical validation 


PSEP (95% CI) 


Model 1 










57 (31 to 84) 


<40% 


159 


22.7% 


24.5% 


75.5% a 




40% to 80% 


114 


56.2% 


54.5% 






>80% 


11 


86.6% 


81.8% 


81.8% b 




Total population 






38.7% 






Model 2 

<40% 


148 


15.7% 


1 9.6% 


80.4% a 


61 (43 to 80) 


40% to 80% 


94 


56.9% 


57.4% 






>80% 


21 


87.1% 


81.0% 


81.0% b 




Total population 






38.0% 







a Negative predictive value for the low-risk group (<40%). b Positive predictive value for the high-risk group (>80%). CI, confidence interval; n, number of patients in 
risk group; PSEP, prognostic separation index. 



useful in our study population. The ECMOnet score has 
been externally validated twice in patient populations with 
a similar profile as the original study [6,14]. Recent results 
indicate that patients receiving ECMO due to influenza- 
A virus infection have a lower mortality than patients 
presenting with other causes of ALF [13]. In addition, 
the ECMOnet study excluded patients with pre-ECMO 
mechanical ventilation >7 days, leaving a patient population 
that may have a more favorable risk profile [4]. Although a 
more homogenous study population improves performance 
and stability of a risk prediction model, this may also limit 
its general applicability. 

The PRESERVE score was developed in a multicenter 
study comprising 140 patients with severe ARDS. The 
additive score (range 0 to 12) consists of eight predictors. 
To our knowledge, this study is the first to externally 
validate the score. Minimal differences in predictor distri- 
butions as well as similar overall mortality rates support 
the comparability of the original and validation study 
populations. There was a clear linear trend in the risk 
of mortality across increasing subgroups of PRESERVE 
score. However, the observed differences in the subgroup 
mortality rates and the reduction in PSEP may be related 
to overfitting from the original analysis [11]. Again, in our 
population the PRESERVE score did not show better 
discrimination than the SOFA score. 

Novel risk models 

Based on the confirmed usefulness of the established 
scores, we identified three strengths in our study which 
justify our attempt to develop a new risk model: First, the 
present study (n = 304) is the largest study investigating 
mortality prediction in ALF patients receiving wECMO 
support. Second, the availability of a broad range of vari- 
ables potentially related to ECMO outcome increases 
the chances of finding an improved combination of pre- 
dictors for mortality following wECMO. Third, the rigor 



of variable selection and the statistical methods applied 
have been designed to avoid overfitting the models to the 
study population [15-17]. 

The final models described in the present study represent 
combinations of variables that contain the most useful 
information to predict ECMO mortality. The models are 
not designed to reflect causal relationships between single 
predictors and outcome, which would only be possible 
if all factors that influence mortality were known. The 
included predictors may not play causal roles themselves, 
but carry information from one or several other predictors 
and thus represent markers for other causal relationships. 

The composition of predictors in Models 1 and 2 under- 
line the importance of the patient's underlying health 
condition and regenerative capacity for prediction of 
ECMO outcome. Advanced age and chronic immuno- 
suppression, both associated with reduced functional 
reserves, high comorbidity and reduced ability to recover, 
have been consistently associated with increased mortality 
[7,9,18]. In accordance with previous findings, we found 
the necessity of high minute ventilation before ECMO 
to be an important predictor of mortality [9]. A high 
pre-ECMO serum lactate concentration could indicate 
tissue hypoxia with subsequent metabolic acidosis and 
severe organ injury. Parallel to reduced hematocrit 
described in the ECMOnet study, low pre-ECMO 
hemoglobin concentrations were associated with an 
increased mortality risk. Possible causes of pre-ECMO 
anemia include hospital-acquired anemia, iron defi- 
ciency anemia or chronic disease. Preoperative anemia 
has repeatedly been described as an independent risk 
factor for increased mortality in cardiac as well as 
noncardiac surgery [19,20]. The harmful effect of 
anemia is greater than the increased risk explained by 
the need for transfusion [20]. Thus, associated comor- 
bid conditions may confound the role of anemia as a 
risk factor. 
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The improved performance with Model 2 indicates 
that ECMO relieves patients from acute respiratory and 
hemodynamic stress and provides time to recover already 
within the first 24 hours. In patients with severe hypox- 
emia or high cardiac output, where ECMO may not 
improve gas exchange sufficiently to decrease aggres- 
siveness of mechanical ventilation, continued invasive 
ventilation with high volume and Fi02 may bring about a 
vicious cycle with ventilator-induced lung injury [4]. Thus, 
sparse reduction or continued need for a high FiC>2 and 
norepinephrine on day 1 may indicate patients with unre- 
sponsive respiratory failure despite ECMO. 

Furthermore, day 1 plasma concentrations of fibrinogen 
and C-reactive protein (CRP) were identified as important 
predictors of ECMO outcome. Surprisingly, low CRP was 
associated with increased risk of mortality. This might be 
related to the ability to activate the immune system in 
order to defend against on-going infection and trigger 
recovery mechanisms. Whereas survivors generally had 
higher CRP before and one day after ECMO implantation, 
they had lower CRP concentrations post-ECMO, leading 
to the hypothesis that survivors were able to quickly acti- 
vate an effective immune response. The lower CRP in 
non-survivors, on the other hand, might indicate immune 
exhaustion or liver failure, with a failing attempt to bring 
about an effective response in order to establish control of 
their acute illness. 

The protective effect seen with higher fibrinogen con- 
centrations may have several explanations. First, it may 
be another sign of an effective immune response, supported 
by the role of fibrinogen as an acute phase reactant. Given 
that both fibrinogen and CRP remained significant, they 
both provide information in addition to that from the other 
factor, even if the two may be somewhat correlated. Second, 
low fibrinogen levels are also associated with bleeding dis- 
orders, large-volume blood transfusions and disseminated 
intravascular coagulation, which are all associated with 
higher mortality. Fibrinogen may thus be an inform- 
ative marker replacing the need for many individual 
markers and may contain information from several 
functions that affect outcome following ECMO support. 

All together, the parameters from Model 2 enhanced dis- 
crimination between patients with high regenerative capacity 
and reversible disease from those with poor health condi- 
tion, reduced functional reserves and intractable illness. 

Challenges related to mortality prediction in vvECMO 
patients 

This study illustrates the difficulties in creating a ro- 
bust model for predicting mortality following wECMO. 
Model 1 showed improved goodness-of-fit and somewhat 
improved discrimination; however, the discriminative abil- 
ity was not significantly enhanced from the PRESERVE 
score. ECMO patients have a large heterogeneity in disease 



and health conditions, and behind 'mortality of all causes' 
there is a large diversity in underlying causality. Patients 
have different diagnoses, time courses, ages, comorbidities, 
different sites and types of infection, and different degrees 
of physiological dysfunction [21]. Therefore, inclusion of 
dayl parameters seemed a promising approach to better 
prediction. The external scores calculated on day 1 data 
did not provide adequate risk prediction, supporting 
the usefulness of our new model. 

It would be of interest to develop a prediction model for 
use later during the course of ECMO, for example on day 
7. Such a model might aid in the decision to wean the 
patient from ECMO. However, due to smaller patient 
numbers caused by earlier weaning or death, a preliminary 
day 7 model for the remaining 193 patients (126 survivors 
and 67 non-survivors) showed wide CIs for the odds ratios 
and was not significantly better at predicting mortality 
than the day 1 model (P = 0.25, data not shown). Thus, it 
seems that a substantially larger study would be needed. 
Furthermore, any model aimed at predicting withdrawal 
of futile therapy has to be extremely accurate in order 
not to lead to erroneous decisions. From our present 
experience with 350 patients, we have observed previously 
unthought-of capacities of injured lungs to recover. 

In this study, different statistical approaches were ex- 
plored and compared in order to derive the best and most 
robust prediction model. While previous models have 
based their variable selection on univariate analyses, we 
employed a method combining clinical experience and 
judgment with computer-based statistics. Nevertheless, 
the inhomogeneous study population makes overfitting a 
persisting challenge and the ability of a predictive model 
to estimate the prognosis of individual patients with high 
accuracy remains limited. The usefulness of a prediction 
model rather lies in the assistance it may provide for the 
discrimination between higher- and lower-risk patients in 
order to help identify potential candidates for ECMO sup- 
port and give guidance for rational and ethical resource 
utilization [6,11]. We, therefore, categorized patients into 
three groups with significantly different probabilities of 
survival. It may be argued that our defined high-risk group 
was very small, but we kept this cut-off because of its 
potential clinical usefulness. Categorization of patients may 
also be helpful for future external and internal validations 
in order to compare institutional variations in indications 
and practice of ECMO. However, although the risk score 
may function as a useful supplementary tool for clinicians, 
thorough clinical evaluation on a case-to-case basis still 
remains the cornerstone of ECMO handling. 

Study limitations 

Conventional rescue therapies, such as prone positioning 
and use of neuromuscular blockade, were not registered 
in all patients in the UKR database. A part of our study 
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population dates back to 2008 when the advantages of 
prone positioning were not universally accepted and 
neuromuscular blockade was only rarely used in Germany. 
Hence, the quality of conventional treatment which may 
be associated with mortality irrespective of the other 
factors included in the model could not be evaluated in 
this study. In the calculation of the PRESERVE scores, all 
patients were assigned to 'no prone positioning pre-ECMO.' 
However, this may contribute to overestimating mortality 
risk with PRESERVE in our patient population. Further- 
more, the PRESERVE score used mortality six months 
post-ICU discharge as the end-point, while the present 
study only followed patients up to hospital discharge. 
Mid- and long-term outcomes are important in ARDS 
patients receiving wECMO support. Unfortunately, in 
the past, long-term outcomes of our patients and health- 
related quality of life were not reevaluated, as many 
patients were retrieved from distant hospitals after our 
team implanted ECMO as a rescue procedure to allow 
transport to our center. We plan to carry out a follow-up 
study to evaluate the validity of the presented models in 
the prediction of mid- and long-term mortality. 

Missing observations due to incomplete documentation 
for some laboratory and respiratory parameters reduced 
the effective sample size in the statistical analyses. Bias 
due to cohort selection or missing data in the presented 
analyses seems unlikely since the overall mortality rate 
was constant in all subsamples with complete information 
for the different parts of the study. Since validation 
through bootstrap methods is preferred over data splitting 
methods [17], we used the whole patient cohort for model 
development. However, external validation is a crucial step 
before these models can be applied in clinical practice. 

Conclusions 

Pre-existing tools for predicting outcome following wEC 
MO in patients with ALF have been of limited usefulness. 
The present study provides two novel risk models designed 
to help assist in defining treatment decision and further 
handling of ECMO. Markers of associated comorbidity, 
a poor pre-ECMO health condition and unresponsive 
respiratory failure were consistently associated with in- 
creased mortality. Mortality prediction improved the day 
following ECMO initiation, when ECMO has reduced acute 
stress and provided time to rest. Nevertheless, due to high 
variability among patients, predictive models can only act 
as supplementary tools and do not replace the role of clin- 
ical evaluation for indication and continuation of ECMO. 

Key messages 

• Established risk assessment tools for adult patients 
with severe acute lung failure on wECMO are of 
limited usefulness. 



• Large patient heterogeneity makes generalized 
pre-ECMO mortality prediction a persisting 
challenge, despite use of robust statistical methods. 

• Addition of variables measured one day after ECMO 
initiation improves mortality prediction. 

• Risk prediction tools function as a useful supplement 
to guide clinical decision-making, although clinical 
evaluation remains the cornerstone of ECMO practice. 

Endnote 

a Available at: http://www.r-project.org. Accessed June 
15, 2013. 
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